In recent years, the amount of information utilized by various organizations, businesses and consumers has exploded to reach enormous amounts. From enterprise resource planning (ERP) to customer resource management (CRM) and other systems, more and more parts of an organization are becoming optimized, thereby producing vast amounts of data relevant to the organization. All of this information needs to be collected, stored, managed, archived, searched and accessed in an efficient, scalable and reliable manner.
Historically, most enterprises have utilized large databases to store the majority of their data and used random access memory (RAM) to locally cache a subset of that data that is most frequently accessed. This has been done mainly to conserve costs since RAM has traditionally been faster but more expensive than disk-based storage. Over time, RAM has been continuously growing in storage capacity and declining in cost. However, these improvements have not kept up with the rapid rate of increase in data being used by enterprises and their numerous applications. In addition, because CPU advancements have generally outpaced memory speed improvements, it is expected that memory latency will become a bottleneck in computing performance.
Organizations today need to predictably scale mission-critical applications to provide fast and reliable access to frequently used data. It is desirable that data be pushed closer to the application for faster access and greater resource utilization. Additionally, continuous data availability and transactional integrity are needed even in the event of a server failure.
An in-memory data grid can provide the data storage and management capabilities by distributing data over a number of servers working together. The data grid can be middleware that runs in the same tier as an application server or within an application server. It can provide management and processing of data and can also push the processing to where the data is located in the grid. In addition, the in-memory data grid can eliminate single points of failure by automatically and transparently failing over and redistributing its clustered data management services when a server becomes inoperative or is disconnected from the network. When a new server is added, or when a failed server is restarted, it can automatically join the cluster and services can be failed back over to it, transparently redistributing the cluster load. The data grid can also include network-level fault tolerance features and transparent soft re-start capability.